Global Multi-view Tracking Utilizing Color and ToF Cameras by Combining Volumetric and Photometric Measures

In this paper a tracking approach designed to utilize multiple cameras with optional depth information, e.g., ToF cameras, structured light cameras and stereo or multi camera setups, is discussed which combines photometric tracking with volumetric tracking. It is able to work with any number and type of cameras. In order to achieve this objective the tracked object is modeled in 3D with an ellipsoid. To make use of the depth information the density of the observed space is modeled with a set of Gaussian kernels for each line of sight. A proposed target configuration is then evaluated by projecting each observed color image onto the ellipsoid and comparing this projection to the expected appearance. Additionally, the density of the space occupied by the ellipsoid is estimated and compared to the expected density. It is demonstrated that by utilizing the depth information in this way ambiguities due to color similarities can be overcome reliably.